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DETAILED ACTION 

Response to Amendment 

1 . The Examiner acknowledges tine amended claims filed on February 23, 201 0. 
Claims 1-5, 7, 11-25, 28-38, 40, 42, 44, 46, 47, 51and 54 have been amended. 
Claims 6, 8-10, 26, 27, 39, 41, 43, 45, 48-50, 52, 53, 55 and 57-60 have been 
cancelled. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1 , 21 , 40, 44, 51 and 54 have been 
considered but are moot in view of the new grounds of rejection. The Applicant further 
argues the following: 

a. "None of the cited references recites generating a saliency signal both to 
indicate user interest in a picture and to control at least one operation of the 
apparatus apart from recording the value of the saliency signal in the memory as 
recited in claim 1 ." 

> The Examiner disagrees. Metcalfe discloses generating saliency signal to 
indicate user interest in a picture {Metcalfe discloses generating a level of interest 
(LOI) set by the user when using button 112) and that at least one operation of the 
apparatus is controlled based on the saliency signal {Metcalfe discloses setting 
the LOI while the camera is activated to take pictures, and storing said LOI 
associated with the images so that said LOI can be used to control the 
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reproduction of the images (i.e. printing, creating thumbnail files for photo albums, 
etc.)) being arranged to be controlled in response to the saliency signal (as 
discussed in page 6, line 31 - page 7, line 8 , Metcalfe discloses that the saliency 
signal (LOI) can be used to control the reproduction of the images (i.e. printing, 
creating thumbnail files for photo albums, etc.)) (See page 4, line 5- page 5, line 
23; page 6, line 23 - page 7, line 8). Therefore, the combined teaching of 
Metcalfe in view of Takahashi teaches the limitations of claim 1 as snow 
presented. 

b. "None of the cited references recites generating a saliency signal both to 
indicate user interest in a picture and to control at least one operation of the 
apparatus to process the image signal as recited in claim 21 ." 

> The Examiner disagrees. As discussed above, Metcalfe discloses 
generating saliency signal to indicate user interest in a picture {Metcalfe 
discloses generating a level of interest (LOI) set by the user when using button 
112) and that at least one operation of the apparatus is controlled based on the 
saliency signal {Metcalfe discloses setting the LOI while the camera is activated 
to take pictures, and storing said LOI associated with the images so that said LOI 
can be used to control the reproduction of the images (i.e. printing, creating 
thumbnail files for photo albums, etc.)) being arranged to be controlled in 
response to the saliency signal (as discussed in page 6, line 31 - page 7, line 8 , 
Metcalfe discloses that the saliency signal (LOI) can be used to control the 
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reproduction of the images (i.e. printing, creating tliumbnail files for piioto 
albums, etc.)) (See page 4, line 5 - page 5, line 23; page 6, line 23 - page 7, line 
8). Therefore, the combined teaching of Metcalfe in view of Takahashi teaches 
the limitations of claim 1 as snow presented. 

c. "None of the cited references recites two or more user controls to receive 
user input to generate first and second saliency signals and combining those 
saliency signals to form a complex saliency signal." 

> The Examiner disagrees. Takahashi teaches circuitry to combine said first 
and second saliency signals while the image signal is being produced to provide 
a composite saliency signal {Takahashi discloses that the auxiliary information 
has persons information and degree of importance of said persons and that the 
auxiliary information is stored with the image signal and further discloses that the 
camera would also display the auxiliary information on a display screen (As 
shown in figs. 23, the user would set the auxiliary information and the set 
auxiliary infonmation would be displayed in the display as shown in figs. 25 and 
26 (see persons information and the degree of importance displayed on the 
display); page 8, ^0118- page 9, H 0124)) (This teaches the combination of the 
two saliency signals while the image signal is being produced (the persons 
information and the degree of importance infonvation stored/displayed together 
in association with the image while the image is being captured upon operation of 
the user) to create a composite saliency signal as claimed. Therefore, by 
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teaching that the person's information and the degree of importance information 
is stored/displayed together in association with the image, Takahashi discloses 
circuitry for combining said first and second saliency signals while the image 
signal is being produced to provide a composite saliency signal as claimed). 
And, as will be discussed in this Office Action, it would have been obvious to one 
of an ordinary skill in the art at the time the invention was made to apply the 
concept of having the camera with a plurality of operation controls to generate 
different saliency signals to be combined to control the operation of the display to 
further displaying the combined saliency signal being assign to the image data as 
taught in Takahashi to modify the teaching of Metcalfe to have at least two 
physically or mechanically operable user controls, each of the user controls for 
generating first and second saliency signals; that said saliency circuitry combines 
said first and second saliency signals to form a composite saliency signal; that 
the operation of at least part of the electronic camera being arranged to be 
controlled in response to the complex saliency signal. The motivation to do so 
would have been to provide the user with a user friendly interface that would 
allow changing the degree of importance of the images being captured and 
would also allow the user to be aware of the information being added to the 
image data. 

d. "None of the cited references recites a combination of 1 ) a user control to 
receive user input to generate a first saliency signal and 2) a saliency circuit to 
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automatically generate an image related second saliency signal, wherein the first 
and second saliency signals are combined to provide a composite saliency 
signal." 

> The Examiner disagrees. As will be discussed in this Office Action, 
Takahashi discloses a physically or mechanically operable user control 
{Takahashi discloses the use of buttons 109 to set auxiliary information (whicli 
the Examiner is interpreting as the saliency information) related to the image data 
being captured; page 6, ^0106. Takahashi further discloses the use of a 
pressure sensor 109a and sweat sensor 109b to determine the auxiliary 
information related to the image data (page 6, ^0112- page 7, U 01 14) to 
receive an input from a user and generate a first saliency signal while the image 
signal is being produced {Takahashi further discloses that the auxiliary 
information includes information related to the persons (i.e. son, daughter, friend, 
father, mother) (see figs. 23(a) and 23 (b); page 8,^0117- page 9, H 0124)) to 
receive an input from a user and to generate, in response to the input from the 
user a saliency signal (auxiliary information having information such as persons 
information)), saliency circuitry to automatically generate an image related 
second saliency signal in response to the image signal {Takahashi further 
teaches that the auxiliary information includes a degree of importance of said 
persons appearing in the image data, and that said degree of importance can be 
determined based on the time length of a scene where a particular person set by 
the user appear; see page 8,^0119. Takahashi further teaches that the level of 
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importance of a particular scene would have a value between 0(0 = not 
important) and 1 (1 = most important) with increments of 0.1 that can be 
determined based on the excitement of the user by using a pressure sensor of a 
sweat sensor or by having a measurement of loudness in the scene being 
captured (See ^0119). Takahashi further discloses that the level of importance 
which can be set manually could also be set automatically by use of said 
pressure, sweat or loudness sensors (Also in H 0119)), and circuitry to combine 
said first and second saliency signals while the image signal is being produced to 
provide a composite saliency signal {Takahashi discloses that the auxiliary 
information has persons information and degree of importance of said persons 
and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary infonmation on a 
display screen (As shown in figs. 23, the user would set the auxiliary information 
and the set auxiliary information would be displayed in the display as shown in 
figs. 25 and 26 (see persons information and the degree of importance displayed 
on the display); page 8, ^0118- page 9, H 0124)) (This teaches the combination 
of the two saliency signals while the image signal is being produced (the persons 
information and the degree of importance information stored/displayed together 
in association with the image while the image is being captured upon operation of 
the user) to create a composite saliency signal as claimed. Therefore, by 
teaching that the person's information and the degree of importance information 
is stored/displayed together in association with the image, Takahashi discloses 
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circuitry for combining said first and second saliency signals while the image 
signal is being produced to provide a composite saliency signal as claimed), 
wherein the first saliency signal, the second saliency signal, and the composite 
saliency signal are to indicate an amount of user interest in the viewed scene 
(Considering that the user sets the first saliency signal and the second saliency 
signal {As discussed above, information related to the persons (i.e. son, 
daughter, friend, father, mother) and the level of importance of a particular scene 
would have a value between 0(0 = not important) and 1 (1 = most important) 
with increments of 0. 1 that can be determined based on the excitement of the 
user by using a pressure sensor of a sweat sensor or by having a measurement 
of loudness in the scene being captured (See H 0119) and as shown in figs. 25 
and 26, the composite saliency signal that is displayed on the screen having the 
first and second saliency signals combined to illustrate the level of importance of 
the scene being produced (page 8, ^0118- page 9, H 0124)) 

e. "None of the cited references recites an apparatus including a buffer 
having a capacity that is to adaptively change in response to a change in the 
value of the saliency signal. 

> The Examiner disagrees. As will be discussed in this Office Action, 
Misawa et al. discloses a memory {Fig. 1: 40) {Page 2, H 0033; page 3, H 0036- 
0040 and H 0043-0044) and having a capacity to store picture signals determined 
in response to the saliency signal {Misawa et al. discloses that that when 



Application/Control Number: 10/696,567 Page 9 

Art Unit: 2622 

capturing images if the priority of the image being captured is high and the 
capacity of the memory 40 is not enough to store the image, images with lower 
priority would be erased to provide space to store the new image having higher 
capacity. In the case that the new captured image has a lower capacity 
compared to the images already stored in memory, said new image with lower 
capacity would not be stored and a message would be displayed indicating that 
no memory capacity is available (Page 3, H 0038-0045). This teaches that the 
memory has a capacity to store picture signals determined in response to the 
saliency signal as claimed since the capacity of the memory is being controlled 
based on the priority given to the images), wherein the capacity of the memory is 
to adaptively change in response to a change in the value of the saliency signal 
(As discussed above, Misawa et al. discloses that when capturing images if the 
priority of the image being captured is high and the capacity of the memory 40 is 
not enough to store the image, images with lower priority would be erased to 
provide space to store the new image having higher capacity. In the case that 
the new captured image has a lower capacity compared to the images already 
stored in memory, said new image with lower capacity would not be stored and a 
message would be displayed indicating that no memory capacity is available 
(Page 3, H 0038-0045). This teaches that the capacity of the memory is to 
adaptively change in response to a change in the value of the saliency signal as 
claimed since the memory is to adaptively change its capacity based on the 
priority selected by the user for a current image (i.e. if the user selected a high 
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priority and the memory is already full, the camera would erase an image with 
normal priority to store the image with high priority). 



f. "None of the cited references recites an apparatus including a picture 
selection circuitry controlled by a saliency signal and a memory to retain or 
discard images based on the saliency signal. 

> The Examiner disagrees. As will be discussed in this Office Action, 
Matsumoto et al. teaches the concept of having a video recording apparatus 

{Fig. 2) of a surveillance system, recording video data captured by an electronic 
camera {See fig. 1), wherein when an alarm is activated, the importance of the 
video is determined to be high as compared to when the alarm is not activated 
{Col. 3, lines 46-67). In col. 4, lines 4, lines 1-19, Matsumoto et al. further 
teaches that based on the degree of importance of the image data being 
recorded, when the memory has no residual capacity images that have lower 
level of importance would be identified and deleted or overwritten so that images 
with high degree of importance would be stored in the storage means {This 
teaches that the memory is to either retain or discard the image based on a value 
of the saliency signal as claimed since the memory would store the image signal 
in place of a stored image when the value of the saliency signal is greater than a 
value of a second saliency signal associated with the stored image and the 
memory is full as claimed). The Examiner understands that it would have been 
obvious to one of ordinary skill in the art a the time the invention was made to 
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apply the concept of replacing innages with lower level of importance with images 
captured with higher levels of importance when the memory has no residual 
capacity, images that have lower level of importance would be identified and 
deleted or overwritten as discussed in Matsumoto et al. to modify the teaching of 
Metcalfe and TakahashI by having the memory retaining or discarding the image 
based on the value of the saliency signal. The motivation to do so would have 
been to record the monitoring image data having a high degree of importance as 
much as possible as suggested in Matsumoto et al. (Co/. 7, lines 35-43). 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the Invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 44, 46 and 47 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Takahashi, US 2002/0041757 Al. 

5. Regarding claim 44, Takahashi discloses an imaging system comprising an 

electronic camera {See figs. 4(a), 4(b), 8, 9(a), 9(b), 10, and 11) to produce an image 
signal representative of a viewed scene, a physically or mechanically operable user 
control {Takaliaslii discloses tlie use of buttons 109 to set auxiliary information (wtiich 
the Examiner is interpreting as the saliency information) related to the image data being 
captured; page 6, H 0^06. Takahashi further discloses the use of a pressure sensor 
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109a and sweat sensor 109b to determine the auxiliary information related to the image 
data (page 6, ^0112- page 7, H 0114) to receive an input from a user and generate a 
first saliency signal while the image signal is being produced {Takahashi further 
discloses that the auxiliary information includes infonmation related to the persons (i.e. 
son, daughter, friend, father, mother) (see figs. 23(a) and 23 (b); page 8, ^0117 - page 
9, H 0124)) to receive an input from a user and to generate, in response to the input 
from the user a saliency signal (auxiliary information having information such as 
persons information)), saliency circuitry to automatically generate an image related 
second saliency signal in response to the image signal {Takahashi further teaches that 
the auxiliary information includes a degree of importance of said persons appearing in 
the image data, and that said degree of importance can be determined based on the 
time length of a scene where a particular person set by the user appear; see page 8, H 
0119. Takahashi further teaches that the level of importance of a particular scene 
would have a value between 0(0 = not important) and 1 (1 = most important) with 
increments of 0. 1 that can be determined based on the excitement of the user by using 
a pressure sensor of a sweat sensor or by having a measurement of loudness in the 
scene being captured (See ^0119). Takahashi further discloses that the level of 
importance which can be set manually could also be set automatically by use of said 
pressure, sweat or loudness sensors (Also in U 0119)), and circuitry to combine said first 
and second saliency signals while the image signal is being produced to provide a 
composite saliency signal {Takahashi discloses that the auxiliary information has 
persons information and degree of importance of said persons and that the auxiliary 
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information is stored with the image signal and further discloses that the camera would 
also display the auxiliary information on a display screen (As shown in figs. 23, the user 
would set the auxiliary information and the set auxiliary information would be displayed 
in the display as shown in figs. 25 and 26 (see persons infonvation and the degree of 
importance displayed on the display); page 8, ^0118- page 9, ^0124)) (This teaches 
the combination of the two saliency signals while the image signal is being produced 
(the persons information and the degree of importance information stored/displayed 
together in association with the image while the image is being captured upon operation 
of the user) to create a composite saliency signal as claimed. Therefore, by teaching 
that the person's information and the degree of importance information is 
stored/displayed together in association with the image, Takahashi discloses circuitry 
for combining said first and second saliency signals while the image signal is being 
produced to provide a composite saliency signal as claimed), wherein tine first saliency 
signal, the second saliency signal, and the composite saliency signal are to indicate an 
amount of user interest in the viewed scene (Considering that the user sets the first 
saliency signal and the second saliency signal {As discussed above, information related 
to the persons (i.e. son, daughter, friend, father, mother) and the level of importance of 
a particular scene would have a value between 0 (0 = not important) and 1 (1 = most 
important) with increments of 0. 1 that can be determined based on the excitement of the 
user by using a pressure sensor of a sweat sensor or by having a measurement of 
loudness in the scene being captured (See H 0119) and as shown in figs. 25 and 26, the 
composite saliency signal that is displayed on the screen having the first and second 
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saliency signals combined to illustrate the level of importance of the scene being 
produced (page 8,^0118- page 9, H 0124)), and the composite saliency signal is to be 
used to control operation of at least a part of the imaging system {Takahashi discloses 
that the auxiliary information has persons information and degree of importance of said 
persons and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary information on a display 
screen (As shown in figs. 23, the user would set the auxiliary information and the set 
auxiliary infomiation would be displayed in the display as shown in figs. 25 and 26 (see 
persons information and the degree of importance displayed on the display); page 8, H 
0118- page 9, ^ 0124)) being arranged to be controlled in response to the composite 
saliency signal (based on the auxiliary information the display would display the 
auxiliary infonmation as shown in figs. 25 and 26). 

6. Regarding claim 46, Takahashi discloses a user operable picture taking control 
to enable the electronic camera to take pictures {shooting button 104 as shown in fig. 4 
(b)). 

7. Regarding claim 47, Takahashi discloses that the first saliency signal is to 
include more than two values {As shown in figs. 23 (a) and 23 (b), Takahashi discloses 
that more than one person can be selected (i.e. the son and the daughter); page 8, H 
0719). 
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Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sl^ill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 1, 3-5, 11-21, 23-25 and 28-38 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Metcalfe, AU 743216 B in view of Takahashi, US 
2002/0041757 A1. 

10. Regarding claim 1, Metcalfe discloses a camera apparatus {See fig. 1) 
comprising an electronic camera (See fig. 1) to produce an image signal, a first user 
operable control {111 as sliown in fig. 1) to selectively activate the electronic camera to 
take pictures, and a second user operable control {112 as shown in fig. 1, note ttiat the 
limitations "an additional physically or mechanically operable user control" are written as 
optional elements by using the word "or") to receive an input from a user and to 
generate, in response to the input from the user a saliency signal to indicate user 
interest in a picture {Metcalfe discloses generating a level of interest (LOI) set by the 
user when using button 112), the saliency signal to have at least one of (a) a value 
selected from at least three different discrete values {Metcalfe discloses assigning a 
level of interest to the image data being recorded, wherein the user can variably assign 
a plurality of level of interest through the capture of the video sequence; see Fig. 3. 
This teaches generating a saliency signal that is selected from at least three different 
discrete values while the image signal is being produced), or (b) a value selected fro a 
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continuous range of values {Metcalfe discloses assigning a level of interest to the image 
data being recorded, wherein the user can variably assign a plurality of level of interest 
through the capture of the a video sequence; see Fig. 3. This teaches generating a 
saliency signal that have a value selected fro a continuous range of values), a circuit to 
record the value of the saliency signal based on the input received via the second 
control contemporaneously with activation of the first control {Metcalfe discloses that the 
LOI is stored as the frames are captured by the video camera 101 (Page 5, lines 8-35) 
and also that after operation of the record button 111, the LOI signals can be input by 
the user and recorded by the video camera (Page 4, lines 5-32). This inherently teaches 
a circuitry to record the value of the saliency signal based on the input received via the 
second control contemporaneously with activation of the first control as claimed since a 
circuitry is necessary to perform the recording operation of the LOI to the tape and also 
considering that the operation to record LOI signals can be performed right after the 
record button to capture image frames is operated as taught in Metcalfe. It is also noted 
that the word "contemporaneous" is defined as "Originating, existing, or happening 
during the same period of time" (Definition from "The American Heritage® Dictionary of 
the English Language", Fourth Edition, 2000)), and a memory arranged to store the 
image signal and the saliency signal {video cassette120 as shown in fig. 1; page 4, lines 
5-13; page 5, lines 4-35), wherein at least one operation of the apparatus is controlled 
based on the saliency signal {Metcalfe discloses setting the LOI while the camera is 
activated to take pictures, and storing said LOI associated with the images so that said 
LOI can be used to control the reproduction of the images (i.e. printing, creating 
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thumbnail files for photo albums, etc.)) being arranged to be controlled in response to 
the saliency signal (as discussed in page 6, line 31 - page 7, line 8 , Metcalfe discloses 
that the saliency signal (LOI) can be used to control the reproduction of the images (i.e. 
printing, creating thumbnail files for photo albums, etc.)) (See page 4, line 5 -page 5, 
line 23; page 6, line 23 - page 7, line 8). 

Metcalfe does not explicitly disclose that the at least one operation is different 
from recording the value of the saliency signal in the memory. 

However, Takahashi discloses an electronic camera {See figs. 4(a), 4(b), 8, 9(a), 
9(b), 10, and 11) producing image signal, comprising a first user operable control 
{shooting button 104 as shown in fig. 4 (b)) to selectively activate the electronic camera 
to take pictures, and a second user operable control {Takahashi discloses the use of 
buttons 109 to set auxiliary infonmation (which the Examiner is interpreting as the 
saliency information) related to the image data being captured; page 6, H 0106. 
Takahashi further discloses the use of a pressure sensor 109a and sweat sensor 109b 
to determine the auxiliary information related to the image data (page 6, ^0112- page 
7, H 0114). Also Takahashi discloses that the auxiliary information includes information 
related to the persons (i.e. son, daughter, friend, father, mother) and a degree of 
importance of said persons appearing in the image data (see figs. 23(a) and 23 (b); 
page 8,^0117 - page 9, H 0124)) to receive an input from a user and to generate, in 
response to the input from the user a saliency signal to indicate user interest in a picture 
{auxiliary information having information such as persons information and degree of 
importance of the persons in the image), the saliency signal to have at least one of (a) a 
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value selected from at least three different discrete values (as shown in figs. 23(a) and 
23(b), tine persons infomiation and degree of importance of the persons in the image 
can change in value between a plurality of values (i.e. three or more values as 
claimed)), or (b) a value selected from a continuous range of values {the auxiliary 
information can be changed during the capture of the image signal to a plurality of 
continuous values as shown in figs. 23(a) and 23(b)), and a memory {Fig. 1: 13) 
arranged to store the image signal and the saliency signal {Takahashi further teaches 
storing the auxiliary information in the header of the scene; page 8,^0123- page 9, H 
0126), wherein at least one operation of the apparatus is controlled based on the 
saliency signal {Takahashi discloses recording the auxiliary information and further 
discloses that the camera would also display the auxiliary information on a display 
screen (As shown in figs. 23, the user would set the auxiliary information and the set 
auxiliary information would be displayed in the display as shown in figs. 25 and 26; page 
8,^0118- page 9, H 0124). It is noted that by teaching that based on the auxiliary 
information the display would display the auxiliary information Takahashi discloses that 
at least one operation of the apparatus is controlled based on the saliency signa\), the at 
least one operation being different from to recording the saliency signal in the memory 
{The Examiner is interpreting displaying the auxiliary information using the display of the 
camera as the operation that is different from recording the saliency signal in the 
memory as claimed). Displaying the auxiliary information while capturing the image 
data is advantageous because it would provide the user with a user friendly interface 
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that would allow changing the degree of importance of the images being captured and 
would also allow the user to be aware of the information being added to the image data. 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of having the camera further displaying the 
salient signal being assign to the image data as taught in Takahashi to modify the 
teaching of Metcalfe to perform an operation in response to the saliency signal, the 
operation being different from recording the saliency signal in the memory. The 
motivation to do so would have been to provide the user with a user friendly interface 
that would allow changing the degree of importance of the images being captured and 
would also allow the user to be aware of the information being added to the image data. 

1 1 . Regarding claim 3, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches a buffer to receive the image signal, 
the buffer having a capacity controlled by the value of the saliency signal {In a further 
embodiment, Takahashi discloses the concept of determining the amount of data to be 
transmitted based on the cost of transmission service; wherein the allowed length of 
data (L) is determined and compared to the amount of data to be transmitted, if the 
amount of data to be transmitted the apparatus would select video section with a priority 
higher than a threshold value. If after selecting those video with the priority higher than 
a threshold, the length of data still more than the allowed length, the apparatus would 
increase the priority threshold, however, if the length is less that the allowed length, the 
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video signals are transmitted (Page 12, H 0151-0154). By teaching adjusting the 
amount of data to be transmitted based on the length allowed for transmission and also 
based on the priority of the video signals to be transmitted, Takahashi inherently 
discloses "a buffer to receive the image signal, the buffer having a capacity controlled 
by the value of the saliency signal" as claimed since the amount being adjusted in the 
apparatus needs to be in a particular buffer or memory prior to transmission in order to 
properly select the video signals with higher priority). One of an ordinary skill in the art 
would have found obvious to apply the concepts of adjusting the amount of data to be 
transmitted based on the priority set to the video signals to further modify the camera 
apparatus to have a buffer for receiving said image signal, the buffer having a capacity 
arranged to be controlled by the value of the saliency signal during operation of the 
camera apparatus with the motivation of select as many as possible the most important 
video signals from the video data for transmission as suggested by Takahashi {Page 
12,^0154). 

1 2. Regarding claim 4, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches image selection circuitry to receive 
the saliency and image signals and to selectively pass the image signal based on the 
saliency signal {As taught in Metcalfe, the camera receives the saliency signal (LOI) and 
based on said saliency signal, when reproducing, the camera would select particular 
images based on the degree of importance as set by the user when recording the 
images. By teaching that the images with a high LOI would be selected for creating an 
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album or to be printed with high quality further teaches that images that do not have a 
high LOI would be passed or skipped since only the images with a high LOI would be 
displayed in the virtual album. Metcalfe as applied reads on ". . .image selection circuitry 
to receive the saliency and image signals and to selectively pass the image signal 

based on the saliency signal" as claimed since the display or skip of images is 
determined based on the LOI set. (See pages 5-7, specifically page 6, line 31 - page 7, 
line 8)). 

1 3. Regarding claim 5, the combined teaching of Metcalfe in view of Tal<ahashi as 
discussed and analyzed in claim 1 further teaches management circuitry to selectively 
retain in the memory {memory 13 in Takahashi, fig. 1) images associated with higher 
saliency levels in preference to images with lower saliency levels {As discussed with 
respect to claim 1, Takahashi further discloses that the images are recorded or 
transmitted based on the importance level of the image, wherein only images with high 
importance level can be recorded in order to reduce the amount of use of the recording 
medium; page 5, H 0094). It would have been obvious to one of an ordinary skill in the 
art at the time the invention was made to modify the teaching of Metcalfe with the 
concepts and teaching of Takahashi to further include management circuitry to 
selectively retain in the memory, images associated with higher saliency levels in 
preference to images with lower saliency levels. The motivation to do so would have 
been to reduce the amount of use of the recording medium as suggested by Takahashi 
{Page 5, H 0094). 
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14. Regarding claim 11, Metcalfe discloses that the first user control includes a 
normal picture taking control on the electronic camera {111 as shown in fig. 1). 

1 5. Regarding claim 12, the combined teaching of Metcalfe in view of Takahashi as 

discussed and analyzed in claim 1 further teaches that the saliency signal is a first 
saliency signal and further comprising a third user operable control to generate a 
second saliency signal {As shown in Takahashi, figs. 4(b): 109; fig. 9(a): 109, 10: 109a, 
and 11: 109b, Takahashi discloses the use of a plurality of buttons to select from 
different auxiliary information to be assigned to the video signal (note that the buttons 
109 are physically located on the camera). Furthermore, as shown in figs 23(a) and 
23(b), Takahashi further discloses that the auxiliary information can be selected using a 
monitor, wherein the user can select the person infonmation and the degree of 
importance of the persons using buttons 103m, 103n, 101m and Win; page 8, ^ 0119 
(although these buttons are displayed on the touch screen display, the Examiner 
understands that the buttons are considered different user controls since they have 
different physical location on said display)). Grounds for rejecting claim 1 apply here. 

16. Regarding claim 13, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 and 12 further teaches saliency circuitry to combine 
the first and second signals to form a complex saliency signal {the Examiner is reading 
the complex saliency signal as the combined information having the persons information 
and the degree of importance information as shown in Takahashi), the complex saliency 
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signal being the saliency signal to control at least one operation {The Examiner is 
reading the operation as the display operation in the camera in Takahashi as discussed 
in claim 1) and the saliency signal stored by the memory {As discussed in claim 1, 
Takahashi discloses that the auxiliary information has persons information and degree 
of importance of said persons and that the auxiliary information is stored with the image 
signal and further discloses that the camera would also display the auxiliary information 
on a display screen (As shown in figs. 23, the user would set the auxiliary information 
and the set auxiliary information would be displayed in the display as shown in figs. 25 
and 26 (see persons information and the degree of importance displayed on the 
display); page 8,^0118- page 9, H 0124)) being arranged to be controlled in response 
to the saliency signal (based on the auxiliary information the display would display the 
auxiliary infonmation as shown in figs. 25 and 26. Furthenmore, the plurality of saliency 
signals are both stored in the memory and the display operation of the camera is 
performed based on the first and second saliency signals as they are input and stored in 
memory). Grounds for rejecting claim 1 and 12 apply here. 

1 7. Regarding claim 14, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches that the saliency signal is a first 
saliency signal {As discussed in claim 1, the saliency signal in Metcalfe for controlling 
the level of interest of a captured scene is considered a first saliency signal as claimed. 
Furthermore, Takahashi further teaches the sue of a plurality of saliency signals to be 
used to control the operation of the camera as discussed in claim 1) and further 
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comprising a saliency circuitry to generate an image related saliency signal in response 
to said image signal {Metcalfe discloses controlling the reproduction operation of the 
camera based on the associated saliency signal to the image signal so that when 
reproducing the image signal with higher importance would be displayed differently from 
the other image signal. See page 4, line 5 - page 5, line 23; page 6, line 23 - page 7, 
line 8. Takahashi further discloses reproducing the video based on the importance level 
of the image, wherein only images with high importance level can be reproduced so that 
the user can enjoy the recorded work without feeling tired, and the power consumption 
is reduced to secure more driving time; page 10, H 0131). 

18. Regarding claim 15, limitations liave been discussed and analyzed in claim 13. 

19. Regarding claim 16, Metcalfe discloses circuitry to incorporate the saliency 
signal in the image signal {Metcalfe discloses incorporating said saliency signal (LOI) to 
each of the frames in the image signal. See page 4, line 5 - page 5, line 23; page 6, 
line 23 - page 7, line 8). 

20. Regarding claim 17, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 teaches that the second user control is part of the 
camera or is physically attached to the camera body {See Metcalfe, controls in Fig. 1; 
see also Takahashi buttons 109 as shown in figs. 4(b), 9(b), 10 and 11, and buttons 
103m, 103n, 101m, 101 n as shown in figs. 23(a) and 23(b)). 
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21 . Regarding claim 18, the combined teaching of Metcalfe in view of Takahashi 
fails to teach that the second user control is a remote control for communication with the 
camera. However, the Examiner takes Official Notice that the concept of controlling a 
camera with a remote control is well known in the art at the time the invention was made 
and that it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have made the user control of Metcalfe and Takahashi a remote 
control as opposed to a camera-body integrated control. One would have been 
motivated to do so because it is well known in the art that by using a remote control to 
control some elements of a camera, the user does not have to be near the camera to 
send and receive desired signals from the camera. This is particularly advantageous in 
cases where plural cameras are used or cameras are placed out of the reach of the 
user (e.g. surveillance cameras), where the remote control would allow the user to send 
signals to the camera(s) from a separate location, thereby simplifying camera control for 
the user. It is noted that in the previous Office Action mailed on November 24, 2009, 
claim 18 was rejected taking Official Notice to the recited limitations. Because the 
Applicant failed to traverse the Examiner's assertion of Official Notice, the well known in 
the art statement is taken to be admitted prior art. See MPEP § 2144.03 [R-1] (C). 

22. Regarding claim 19, Metcalfe discloses that the second user control comprises 
a physically movable control member {pressure button that assign a level of interest of 
an image signal based on the pressure applied to said button) and a sensor arranged to 
be responsive to movement of the control member (See page 4, line 27 - page 5, line 



Application/Control Number: 10/696,567 Page 26 

Art Unit: 2622 

4). By teaching a pressure button that assigns a LOI of an image signal based on the 
pressure applied to the button, Metcalfe discloses a physically movable control member 
(in this case the pressure button as discussed in Metcalfe) and a sensor arranged to be 
responsive to movement of the control member since by teaching that the LOI is 
assigned based on the pressure applied to the button, the use of a sensor to determine 
the applied pressure in order to assigned the LOI of an image is inherent and 
necessitated in Metcalfe). 

23. Regarding claim 20, the Examiner notes that the limitations "the second user 
control comprises a pressure or force sensing transducer for deriving the saliency 
signal that can have values that are continuously variable", the elements "pressure" or 
"force sensing transducer" are written as optional elements by using the word "or". 

IVIetcalfe discloses that the second user control comprises a pressure or force 
sensing transducer for deriving the saliency signal that can have values that are 
continuously variable {Metcalfe as applied to claim 19, teaches the use of pressure 
(pressure button that assign a level of interest of an image signal based on the pressure 
applied to said button; page 4, line 27 - page 5, line 4) to determine the value of the 
saliency signal. By teaching that Metcalfe discloses the use of pressure applied to the 
button, Metcalfe discloses the use of pressure to derive the value of the saliency signal 
as claimed. 



24. 



Regarding claim 21, limitations have been discussed and analyzed in claim 1 . 
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25. Regarding claim 23, limitations have been discussed and analyzed in claim 4, 
noting that the limitations "...selectively pass or delete" are written as optional. 

26. Regarding claim 24, limitations have been discussed and analyzed in claim 3. 

27. Regarding claim 25, limitations have been discussed and analyzed in claim 5. 

28. Regarding claim 28, limitations have been discussed and analyzed in claim 1 . 
Note that Metcalfe and Takahashi, both teach user operable controls to generate the 
Image signal (Metcalfe discloses an user operable control {111 as shown in fig. 1) to 
selectively activate the electronic camera to take pictures, and another user operable 
control to receive an input from a user and to generate, in response to the input from the 
user a saliency signal to indicate user interest in a picture {Metcalfe discloses 
generating a level of interest (LOI) set by the user when using button 112). Takahashi 
discloses a user operable control {shooting button 104 as shown in fig. 4 (b)) to 
selectively activate the electronic camera to take pictures, and another user operable 
control {Takahashi discloses the use of buttons 109 to set auxiliary information). 

29. Regarding claim 29, limitations have been discussed and analyzed in claim 1 1 . 
Furthermore Metcalfe discloses that the saliency signal can be produced by the same 
button for capturing video {Page 4, lines 27-30). 
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30. Regarding claim 30, limitations have been discussed and analyzed in claim 12. 



31 . Regarding claim 31, limitations have been discussed and analyzed in claim 13. 

32. Regarding claim 32, limitations have been discussed and analyzed in claim 13. 

33. Regarding claim 33, limitations have been discussed and analyzed in claims 13 
and 14. 

34. Regarding claim 34, limitations have been discussed and analyzed in claim 16. 

35. Regarding claim 35, limitations have been discussed and analyzed in claim 17. 

36. Regarding claim 36, limitations have been discussed and analyzed in claim 18. 

37. Regarding claim 37, limitations have been discussed and analyzed in claim 19. 

38. Regarding claim 38, limitations have been discussed and analyzed in claims 19 
and 20. 



Application/Control Number: 10/696,567 Page 29 

Art Unit: 2622 

39. Claims 2, 7, 22, 40, 42, 54 and 56 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Metcalfe, AU 743216 B in view of Takahashi, US 
2002/0041757 A1 and further in view of Matsumoto et al., US Patent 6,795,642 B2. 

40. Regarding claim 2, although Takahashi teaches a compression circuitry {See 
Takahashi, fig. 1: 15; page 5, H 0097; page 6, H 0105) to compress the image signals 
and that although the invention is described on the premise that a shot picture is 
recorded, a shot picture is not necessarily recorded, and it can be used also when 
compressed video and audio data are transmitted as they are to be used on a network 
or the like {page 10, H 0130), the combined teaching of Metcalfe in view of Takahashi 
fails to teach compressing the image signals to an extent determined by the saliency 
signal. 

However, Matsumoto et al. teaches the concept of having a video recording 

apparatus {Fig. 2) of a surveillance system, recording video data captured by an 
electronic camera {See fig. 1), wherein when an alarm is activated, the importance of 
the video is determined to be high as compared to when the alarm is not activated (Co/. 
3, lines 46-67). Matsumoto et al. further discloses that based on the degree of 
importance given to the video signal, the data compression is also adjusted {i.e. if the 
importance degree of the video is low, it would be compressed at high level and if the 
importance degree of the video is high, said video would be compressed at low level) 
{Col. 3, line 15- col. 4, lines 19). Matsumoto also discloses that the importance level 
can also be adjusted by the user operating the surveillance system {Col. 7, lines 22-34). 
Matsumoto et al. further discloses that by adjusting the compression of the video being 
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captured, it is possible to record the monitoring image data having a high degree of 
importance as much as possible (Co/. 7, lines 35-43). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi and 
further in view of Matsumoto et al. as a whole, it would have been obvious to one of 
ordinary skill in the art a the time the invention was made to apply the concept of 
adjusting the compression of a video captured by an electronic camera based on a 
degree of importance assign to the video as discussed in Matsumoto et al. to modify the 
teaching of Metcalfe and Takahashi by compressing the image signals to an extent 
determined by the saliency signal. The motivation to do so would have been to record 
the monitoring image data having a high degree of importance as much as possible as 
suggested in Matsumoto et al. (Co/. 7, lines 35-43). 

41 . Regarding claim 7, the combined teaching of Metcalfe in view of Takahashi and 
further in view of Matsumoto et al. further teaches a management circuitry to selectively 
retaining in the memory images associated with higher saliency levels in preference to 
images with lower saliency levels (Takahashi further discloses that the images are 

recorded or transmitted based on the importance level of the image, wherein only 
images with high importance level can be recorded in order to reduce the amount of use 
of the recording medium; page 5, H 0094). 

42. Regarding claim 22, limitations have been discussed and analyzed in claim 2. 
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43. Regarding claim 40, IVIetcalfe discloses an imaging system (See fig. 1) 
comprising an electronic camera (See fig. 1) to produce an image signal, physically or 
mechanically operable user controls {See user controls 106, 104, 111, 112, 110 and 
108 as sliown in fig. 1), the user controls being arranged to receive an input from a user 
and to generate first saliency signal {Metcalfe discloses the use of button 112 to 
generate a plurality of saliency signals (Level of interest signals "LOI") to be associated 
to the image signal being recorded with the camera) while the image signal is being 
produced, and saliency circuitry {the camera in Metcalfe inherently has a saliency signal 
circuitry to generate the saliency signal upon operation of the camera button 112) for 
storing said first saliency signal {Metcalfe discloses recording the plurality of saliency 
signals in a memory (tape 120 in fig. 1) in association with the image data; see page 4, 
line 5 - page 5, line 23; page 6, line 23 - page 7, line 8), the saliency signal having at 
least one of (a) a value selected from at least three different discrete values {Metcalfe 
discloses assigning a level of interest to the image data being recorded, wherein the 
user can variably assign a plurality of level of interest through the capture of the a video 
sequence; see Fig. 3. This teaches generating a saliency signal having a value from at 
least three different discrete values), or (b) a value selected from a continuous range of 
values {Metcalfe discloses assigning a level of interest to the image data being 
recorded, wherein the user can variably assign a plurality of level of interest through the 
capture of the a video sequence; see Fig. 3. This teaches generating a saliency signal 
having a value selected from a continuous range of values), a memory {120 as shown in 
fig. 1) to store the image signal and the saliency signal in response to the saliency 
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signal (page 4, lines 5-13; page 5, lines 4-35), and operation of at least part of the 
electronic camera being arranged to be controlled in response to the saliency signal (as 
discussed in page 6, line 31 - page 7, line 8, Metcalfe discloses that the saliency signal 
(LOI signal stored in memory 120) can be used to control the reproduction of the 
images (i.e. printing, creating thumbnail files for photo albums, etc.)) (See page 4, line 5 
- page 5, line 23; page 6, line 23 - page 7, line 8). 

Metcalfe does not explicitly disclose at least two physically or mechanically 
operable user controls, each of the user controls for generating first and second 
saliency signals; that said saliency circuitry combines said first and second saliency 
signals to form a complex saliency signal; and that said memory stores the image signal 
in place of a stored image when the value of the saliency signal is greater than a value 
of a second saliency signal associated with the stored image and the memory is full. 

However, Takahashi teaches an imaging system comprising an electronic 
camera {See figs. 4(a), 4(b), 8, 9(a), 9(b), 10, and 11) to produce an image signal 
representative of a viewed scene, a physically or mechanically operable user control 
[Takahashi discloses the use of buttons 109 to set auxiliary information (which the 
Examiner is interpreting as the saliency information) related to the image data being 
captured; page 6, ^0106. Tal<ahashi further discloses the use of a pressure sensor 
109a and sweat sensor 109b to determine the auxiliary information related to the image 
data (page 6, ^0112- page 7, H 0114) to receive an input from a user and generate a 
first saliency signal while the image signal is being produced {Takahashi further 
discloses that the auxiliary information includes information related to the persons (i.e. 
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son, daughter, friend, father, mother) (see figs. 23(a) and 23 (b); page 8, ^0117 - page 
9, H 0124)) to receive an input from a user and to generate, in response to the input 
from the user a saliency signal (auxiliary information having information such as 
persons information)), saliency circuitry to automatically generate an image related 
second saliency signal in response to the image signal {Tal<ahashi further teaches that 
the auxiliary information includes a degree of importance of said persons appearing in 
the image data, and that said degree of importance can be determined based on the 
time length of a scene where a particular person set by the user appear; see page 8, H 
01 19. Takahashi further teaches that the level of importance of a particular scene 
would have a value between 0(0 = not important) and 1 (1 = most important) with 
increments of 0. 1 that can be determined based on the excitement of the user by using 
a pressure sensor of a sweat sensor or by having a measurement of loudness in the 
scene being captured (See ^ 01 19). Takahashi further discloses that the level of 
importance which can be set manually could also be set automatically by use of said 
pressure, sweat or loudness sensors (Also in H 0119)), and circuitry to combine said first 
and second saliency signals while the image signal is being produced to provide a 
composite saliency signal {Takahashi discloses that the auxiliary information has 
persons information and degree of importance of said persons and that the auxiliary 
information is stored with the image signal and further discloses that the camera would 
also display the auxiliary information on a display screen (As shown in figs. 23, the user 
would set the auxiliary information and the set auxiliary information would be displayed 
in the display as shown in figs. 25 and 26 (see persons infonvation and the degree of 
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importance displayed on the display); page 8, ^0118- page 9, H 0124)) (Tliis teaches 
the combination of the two saliency signals while the image signal is being produced 
(the persons information and the degree of importance information stored/displayed 
together in association with the image while the image is being captured upon operation 
of the user) to create a composite saliency signal as claimed. Therefore, by teaching 
that the person's information and the degree of importance information is 
stored/displayed together in association with the image, Takahashi discloses circuitry 
for combining said first and second saliency signals while the image signal is being 
produced to provide a composite saliency signal as claimed), wherein the first saliency 
signal, the second saliency signal, and the composite saliency signal are to indicate an 
amount of user interest in the viewed scene (Considering that the user sets the first 
saliency signal and the second saliency signal {As discussed above, information related 
to the persons (i.e. son, daughter, friend, father, mother) and the level of importance of 
a particular scene would have a value between 0(0 = not important) and 1 (1 = most 
important) with increments of 0. 1 that can be determined based on the excitement of the 
user by using a pressure sensor of a sweat sensor or by having a measurement of 
loudness in the scene being captured (See H 0119) and as shown in figs. 25 and 26, the 
composite saliency signal that is displayed on the screen having the first and second 
saliency signals combined to illustrate the level of importance of the scene being 
produced (page 8, ^0118- page 9, H 0124)), and the composite saliency signal is to be 
used to control operation of at least a part of the imaging system {Takahashi discloses 
that the auxiliary information has persons information and degree of importance of said 
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persons and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary infomnation on a display 
screen (As shown in figs. 23, the user would set the auxiliary information and the set 
auxiliary infomnation would be displayed in the display as shown in figs. 25 and 26 (see 
persons information and the degree of importance displayed on the display); page 8, H 
0118- page 9, H 0124)) being arranged to be controlled in response to the composite 
saliency signal (based on the auxiliary information the display would display the 
auxiliary infomnation as shown in figs. 25 and 26). 

Therefore, taking tine combined teaching of Metcalfe in view of Tal<ahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of having the camera with a plurality of 
operation controls to generate different saliency signals to be combined to control the 
operation of the display to further displaying the combined saliency signal being assign 
to the image data as taught in Takahashi to modify the teaching of Metcalfe to have at 
least two physically or mechanically operable user controls, each of the user controls for 
generating first and second saliency signals; that said saliency circuitry combines said 
first and second saliency signals to form a composite saliency signal; that the operation 
of at least part of the electronic camera being arranged to be controlled in response to 
the complex saliency signal. The motivation to do so would have been to provide the 
user with a user friendly interface that would allow changing the degree of importance of 
the images being captured and would also allow the user to be aware of the information 
being added to the image data. 
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The combined teacliing of Metcalfe in view of Takahashi fails to teach that the 
memory stores the image signal in place of a stored image when the value of the 
saliency signal is greater than a value of a second saliency signal associated with the 
stored image and the memory is full. 

However, Matsumoto et al. teaches the concept of having a video recording 
apparatus {Fig. 2) of a surveillance system, recording video data captured by an 
electronic camera (See fig. 1), wherein when an alarm is activated, the importance of 
the video is determined to be high as compared to when the alarm is not activated (Co/. 
3, lines 46-67). Matsumoto et al. further discloses that based on the degree of 
importance given to the video signal, the data compression is also adjusted {i.e. if the 
importance degree of the video is low, it would be compressed at high level and if the 
importance degree of the video is high, said video would be compressed at low level) 
{Col. 3, line 15- col. 4, lines 19). Matsumoto also discloses that the importance level 
can also be adjusted by the user operating the surveillance system {Col. 7, lines 22-34). 
Matsumoto et al. further discloses that by adjusting the compression of the video being 
captured, it is possible to record the monitoring image data having a high degree of 
importance as much as possible {Col. 7, lines 35-43). In col. 4, lines 4, lines 1-19, 
Matsumoto et al. further teaches that based on the degree of importance of the image 
data being recorded, when the memory has no residual capacity images that have lower 
level of importance would be identified and deleted or overwritten so that images with 
high degree of importance would be stored in the storage means (This teaches that the 
memory stores the image signal in place of a stored image when the value of the 



Application/Control Number: 10/696,567 Page 37 

Art Unit: 2622 

saliency signal is greater tlian a value of a second saliency signal associated with the 
stored image and the memory is full as claimed). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi and 
further in view of Matsumoto et al. as a whole, it would have been obvious to one of 
ordinary skill in the art a the time the Invention was made to apply the concept of 
replacing images with lower level of importance with images captured with higher levels 
of importance when the memory has no residual capacity, images that have lower level 
of importance would be identified and deleted or overwritten as discussed in Matsumoto 
et al. to modify the teaching of Metcalfe and Takahashi by having the memory storing 
the image signal in place of a stored image when the value of the saliency signal is 
greater than a value of a second saliency signal associated with the stored image and 
the memory is full. The motivation to do so would have been to record the monitoring 
image data having a high degree of importance as much as possible as suggested in 
Matsumoto et al. (Co/. 7, lines 35-43). 

44. Regarding claim 42, the combined teaching of Metcalfe in view of Takahashi 
and further in view of Matsumoto et al. as discussed and analyzed in claim 40 further 
teaches a separate user operable picture taking control for selectively activating the 
electronic camera to take pictures {As shown in Takaliaslii, figs. 4(b): 109; fig. 9(a): 109, 
10: 109a, and 11: 109b, Takaliaslii discloses the use of a plurality of buttons to select 
from different auxiliary information to be assigned to the video signal (note that the 
buttons 109 are physically located on the camera). Furthermore, as shown in figs 23(a) 
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and 23(b), Takahashi further discloses that the auxiliary information can be selected 
using a monitor, wherein the user can select the person information and the degree of 
importance of the persons using buttons 103m, 103n, 101m and Win; page 8, H 0119 
(although these buttons are displayed on the touch screen display, the Examiner 
understands that the buttons are considered different user controls since they have 
different physical location on said display)). 

45. Regarding claim 54, IVIetcalfe discloses an apparatus {See fig. 1) comprising an 

electronic camera (See fig. 1) having a picture taking control (button 111 as shown in 
fig. 1) to selectively activate the camera to store an image to a memory {Fig. 1: 120), the 
camera further including a user operable control {button 112 as shown in fig. 1) to 
generate a non-playback saliency signal {the saliency signal generated in Metcalfe is a 
level of interest signal to indicate portion of the image signals that have certain degree 
of interest to be stored in the memory 120 in association with the saliency signal) and 
picture selection circuitry {circuitry is inherent in the Metcalfe reference to control the 
recording and reproduction of the video signals stored in the memory 120) to selectively 
pass the picture signals in response to the saliency signal {Metcalfe discloses that the 
user can select a saliency signal (LOI) to a particular image being captured, the saliency 
signal is stored in association with the image (See pages 5-7). Metcalfe further 
discloses that based on the stored saliency signal, the images would be reproduced, 
wherein when reproducing, the camera would select particular images based on the 
degree of importance (LOI) as set by the user when recording the images (See pages 
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5-7; page 6, line 31 - page 7, line 8)). By teaching that the images with a high LOI 
would be selected for creating an album or to be printed with high quality further 
teaches that images that do not have a high LOI would be passed or skipped since only 
the images with a high LOI would be displayed in the virtual album. Metcalfe as applied 
reads on the limitations ". . . picture selection circuitry for selectively passing the picture 
signals in response to the saliency signal ..."since the display or skip of images is 
determined based on the saliency signal (LOI)) (See page 4, line 5 - page 5, line 23; 
page 6, line 23 - page 7, line 8). 

Although Metcalfe discloses selectively passing the image signals during 
reproduction of the video based on the saliency signal, Metcalfe does not explicitly 
disclose selectively passing the image to the memory in response to the saliency signal 
and that the memory is to retain or discard the image based on a value of the saliency 
signal. 

However, Takahashi teaches an imaging system comprising an electronic 
camera {See figs. 4(a), 4(b), 8, 9(a), 9(b), 10, and 11) to produce an image signal 
representative of a viewed scene, a physically or mechanically operable user control 
{Takahashi discloses the use of buttons 109 to set auxiliary information (which the 
Examiner is interpreting as the saliency information) related to the image data being 
captured; page 6, ^0106. Takahashi further discloses the use of a pressure sensor 
109a and sweat sensor 109b to determine the auxiliary information related to the image 
data (page 6, ^0112- page 7, H 0114) to receive an input from a user and generate a 
first saliency signal while the image signal is being produced {Takahashi further 
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discloses that the auxiliary information includes information related to the persons (i.e. 
son, daughter, friend, father, mother) (see figs. 23(a) and 23 (b); page 8, ^0117 - page 
9, H 0124)) to receive an input from a user and to generate, in response to the input 
from the user a saliency signal (auxiliary infomnation having information such as 
persons information)), saliency circuitry to automatically generate an image related 
second saliency signal in response to the image signal {Takahashi further teaches that 
the auxiliary information includes a degree of importance of said persons appearing in 
the image data, and that said degree of importance can be determined based on the 
time length of a scene where a particular person set by the user appear; see page 8, H 
01 19. Takahashi further teaches that the level of importance of a particular scene 
would have a value between 0(0 = not important) and 1 (1 = most important) with 
increments of 0. 1 that can be determined based on the excitement of the user by using 
a pressure sensor of a sweat sensor or by having a measurement of loudness in the 
scene being captured (See H 01 19). Takahashi further discloses that the level of 
importance which can be set manually could also be set automatically by use of said 
pressure, sweat or loudness sensors (Also in H 0119)), and circuitry to combine said first 
and second saliency signals while the image signal is being produced to provide a 
composite saliency signal {Takahashi discloses that the auxiliary information has 
persons information and degree of importance of said persons and that the auxiliary 
information is stored with the image signal and further discloses that the camera would 
also display the auxiliary information on a display screen (As shown in figs. 23, the user 
would set the auxiliary information and the set auxiliary information would be displayed 
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in the display as shown in figs. 25 and 26 (see persons information and the degree of 
importance displayed on the display); page 8,^0118- page 9,^0124)) (This teaches 
the combination of the two saliency signals while the image signal is being produced 
(the persons infomiation and the degree of importance information stored/displayed 
together in association with the image while the image is being captured upon operation 
of the user) to create a composite saliency signal as claimed. Therefore, by teaching 
that the person's information and the degree of importance information is 
stored/displayed together in association with the image, Takahashi discloses circuitry 
for combining said first and second saliency signals while the image signal is being 
produced to provide a composite saliency signal as claimed), wherein tlie first saliency 
signal, the second saliency signal, and the composite saliency signal are to indicate an 
amount of user interest in the viewed scene (Considering that the user sets the first 
saliency signal and the second saliency signal {As discussed above, information related 
to the persons (i.e. son, daughter, friend, father, mother) and the level of importance of 
a particular scene would have a value between 0 (0 = not important) and 1 (1 = most 
important) with increments of 0. 1 that can be determined based on the excitement of the 
user by using a pressure sensor of a sweat sensor or by having a measurement of 
loudness in the scene being captured (See H 0119) and as shown in figs. 25 and 26, the 
composite saliency signal that is displayed on the screen having the first and second 
saliency signals combined to illustrate the level of importance of the scene being 
produced (page 8, ^0118- page 9, H 0124)), and the composite saliency signal is to be 
used to control operation of at least a part of the imaging system {Takahashi discloses 
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that the auxiliary information has persons information and degree of importance of said 
persons and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary information on a display 
screen (As shown in figs. 23, the user would set the auxiliary information and the set 
auxiliary information would be displayed in the display as shown in figs. 25 and 26 (see 
persons information and the degree of importance displayed on the display); page 8, H 
0118- page 9, H 0124)) being anranged to be controlled in response to the composite 
saliency signal (based on the auxiliary information the display would display the 
auxiliary information as shown in figs. 25 and 26). Also, Takahashi teaches that the 
important images from a captured video can be extracted so that only scenes of high 
degrees of importance can be efficiently recorded or transmitted as compared with the 
case where all of the shot scenes are stored or transmitted and that the extraction of the 
scenes of high degrees of importance may be carried out during or after shooting of the 
scenes {See page 5, H 0094; page 10,^ 0130) {This teaches the concept of selectively 
passing the image to the memory in response to the saliency signal and that the 
memory is to retain or discard the image based on the value of the saliency signal as 
claimed since the image signals would be either recorded or transmitted based in the 
priority given to the scenes so that only important scenes are transmitted or recorded 
while said scenes are being captured). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of selectively recording image data in a 
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memory in response to an assigned priority to scenes in tlie image data as taught in 
Tal<ahashi to modify the teaching of Metcalfe to selectively pass the image to the 
memory in response to the saliency signal. The motivation to do so would have been to 
reduce the amount of use in the recording medium and reduce the communication costs 
as suggested by TakahashI {Page 5, H 0094). 

The combined teaching of Metcalfe in view of Takahashi fails to teach that the 
memory is to retain or discard the image based on the value of the saliency signal. 

However, Matsumoto et al. teaches the concept of having a video recording 
apparatus {Fig. 2) of a surveillance system, recording video data captured by an 
electronic camera {See fig. 1), wherein when an alarm is activated, the importance of 
the video is determined to be high as compared to when the alarm is not activated {Col. 
3, lines 46-67). Matsumoto et al. further discloses that based on the degree of 
Importance given to the video signal, the data compression Is also adjusted (/. e. if the 
importance degree of the video is low, it would be compressed at high level and if the 
importance degree of the video is high, said video would be compressed at low level) 
{Col. 3, line 15- col. 4, lines 19). Matsumoto also discloses that the importance level 
can also be adjusted by the user operating the surveillance system {Col. 7, lines 22-34). 
Matsumoto et al. further discloses that by adjusting the compression of the video being 
captured, It Is possible to record the monitoring image data having a high degree of 
importance as much as possible (Co/. 7, lines 35-43). In col. 4, lines 4, lines 1-19, 
Matsumoto et al. further teaches that based on the degree of importance of the image 
data being recorded, when the memory has no residual capacity images that have lower 
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level of importance would be identified and deleted or overwritten so that images with 
high degree of importance would be stored in the storage means {This teaches that the 
memory is to either retain or discard the image based on a value of the saliency signal 
as claimed since the memory would store the image signal in place of a stored image 
when the value of the saliency signal is greater than a value of a second saliency signal 
associated with the stored image and the memory is full as claimed). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi and 
further in view of Matsumoto et al. as a whole, it would have been obvious to one of 
ordinary skill in the art a the time the invention was made to apply the concept of 
replacing images with lower level of importance with images captured with higher levels 
of importance when the memory has no residual capacity, images that have lower level 
of importance would be identified and deleted or overwritten as discussed in Matsumoto 
et al. to modify the teaching of Metcalfe and Takahashi by having the memory retaining 
or discarding the image based on the value of the saliency signal. The motivation to do 
so would have been to record the monitoring image data having a high degree of 
importance as much as possible as suggested in Matsumoto et al. (Co/. 7, lines 35-43). 

46. Regarding claim 56, limitations have been discussed and analyzed in claim 54. 
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47. Claim 51 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Misawa et al., US 2002/0118285 A1 in view of Hayashi, US 2002/0031349 A1. 

48. Regarding claim 51 , Misawa et al. discloses an apparatus comprising an 
electronic camera {See figs. 1 and 2) having a picture taking control to selectively 

activate the camera {shutter button 14 as shown in fig. 2), a user operable control 
{priority selection switch 16 as shown in fig. 2; page 2, H 0026-0029 and U 0035; page 3, 
H 0037-0044) to generate a saliency signal when the picture taking control activates the 
camera {Misawa et al. discloses generating a particular priority given to the image 
based on the selection made by the user (page 2, H 0026-0029 and H 0035; page 3, H 
0037-0044)), the saliency signal to include plural values {Note that the Examiner is 
interpreting the priority given to the images as the saliency signal as claimed which 
contains difference priority values such as nonmal, high and memo as shown in fig. 3. 
See page 2, H 0027) and a memory {Fig. 1: 40) {Page 2, H 0033; page 3, H 0036-0040 
and H 0043-0044) and having a capacity to store picture signals determined in response 
to the saliency signal {Misawa et al. discloses that that when capturing images if the 
priority of the image being captured is high and the capacity of the memory 40 is not 
enough to store the image, images with lower priority would be erased to provide space 
to store the new image having higher capacity. In the case that the new captured image 
has a lower capacity compared to the images already stored in memory, said new 
image with lower capacity would not be stored and a message would be displayed 
indicating that no memory capacity is available (Page 3, H 0038-0045). This teaches 
that the memory has a capacity to store picture signals determined in response to the 
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saliency signal as claimed since tlie capacity of the memory is being controlled based 
on ttie priority given to tine images), wherein tlie capacity of the memory is to adaptively 
change in response to a change in the value of the saliency signal (As discussed above, 
Misawa et al. discloses that when capturing images if the priority of the image being 
captured is high and the capacity of the memory 40 is not enough to store the image, 
images with lower priority would be erased to provide space to store the new image 
having higher capacity. In the case that the new captured image has a lower capacity 
compared to the images already stored in memory, said new image with lower capacity 
would not be stored and a message would be displayed indicating that no memory 
capacity is available (Page 3, H 0038-0045). This teaches that the capacity of the 
memory is to adaptively change in response to a change in the value of the saliency 
signal as claimed since the memory is to adaptively change its capacity based on the 
priority selected by the user for a current image (i.e. if the user selected a high priority 
and the memory is already full, the camera would erase an image with normal priority to 
store the image with high priority). Misawa et al. further discloses that although the 
invention has been discussed with reference to still images, the invention can also be 
applied to motion images {Page 4, ^0063). 

Misawa et al. does not explicitly disclose that said memory is a buffer memory. 

However, Hayashi discloses an electronic camera {See fig. 1) comprising a 
picture taking control for selectively activating the camera to derive input picture signal 
{Operating member 27 as shown in fig. 1; page 3, H 0050) and a buffer memory for 
receiving the input picture signals {Hayashi teaches that the microprocessor 25 includes 
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a buffer that is capable of adjust its capacity in accordance with the requirements to 
meet a god display resolution and the minimum number of continuous shots to be 
guaranteed (Page 3, H 0054; page 4, H 0080)) and having a capacity for the input 
picture signals determined in response to the needs for properly store/display the image 
data {As discussed before, Hayashi discloses that the capacity of the buffer is modified 
to allow proper display and storage of the image data being captured so that the 
capacity of the buffer is either reduced or increased accordingly (Pages 4-5, H 0076- 
0026)). Hayashi further teaches that by adjusting the size or capacity of the buffer in 
accordance to the data being captured, an improvement on the timing for processing 
image is attained so that shooting and recording time of images is reduced {Page 5, H 
0097-0098). 

Therefore, taking the combined teaching of Misawa et al. in view of Hayashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the teaching of using a buffer that has a capacity that can 
be adjusted as taught in Hayashi to modify said memory in Misawa et al. to use a buffer 
to store the image signals as an alternative to the memory 40. The motivation to do so 
would have been to improve the timing for processing image is attained so that shooting 
and recording time of images is reduced as suggested by Hayashi {Page 5, H 0097- 
0098). 
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Conclusion 

49. Applicant's amendment necessitated tine new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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